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ABSTBACT 

This docuaent ezaaines contextual variables and hov 
they af£ect instructional processes and educational outcoaes. One 
contextual variable which has not received enough attention is the 
curriculua*naterial package, curriculua packages represent potential 
experiaental treatnents. Possible research guestions are whether 
these materials and instructions are suitable, whether they are used 
properly, and whether the outcoaes are the ones which are expected. 
Once the effectiveness of a curriculua package is analyzed, the next 
research step is to assess the content and how it affects curriculum 
results. Besearch to date indicates that different curriculuas 
produce different patterns of achieveaent and that these patterns are 
strongly influenced by the content emphasized in the curriculum. 
Further research into this area might study the influence of content, 
time available for instruction, student attention to task, 
opportunity to learn, and teacher emphasis on student achievement. 
(Author/DB) 
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In this piqper a nua2»er of topics on the possible io^iortance of a 
corriculua package are explored. The first topic is a discussion of 
contextual variables, follwed by an aegusBBnt for the importance of a 
curri&tilua package in aiding instruction. Next is a discussion of hew 
lauMflc<lge of curriculum could be used in process outcome research and 
a discussion of previously neglected variabletf such as the amount of 
help which a curriculum mi^t provide for a teacher. The paper concludes 
with a discussion of two related topics: the importance of considering 
cratent covered and the importance of doing research to elirinate 
unproductive research ideas. 

The Hwsbet of Contextual Variables 

The argument for considering contextual variables is that the 
correlation between instrwtional processes and educational outcomes 
will be different when one is stratified by the proposed contextual 
variable. For example, it is frequently suggested that if classrooms 
were stratified by the pupils' socioeconoinic status the correlations 
between instructional processes end student outcomes would be differ^t 
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wlthln each SES level. Similarly, it is suggested that correlations 
will differ across grade levels* across subject areas » according to 
the outcome measure being used, according to the location of the school 
(e.g. urban, rural, suburban), according to the SES status of the pupils, 
and according to any number of suggested contextual variables. 

Ihe potential in^ortence of context can operate in a tmhet of 
ways. Context can be a stratifying variable in that certain process- 
outcome correlational relationships may be stronger or weaker in dif- 
ferent contextual combinations. It can also operate as a facilitative 
variable in an experiment. In such cases, one studies Whether certain 
acquired teaching skills (or certain curriculum programs) operate more 
effectively in one context or another. The effects of training teachers 
in asking higher level questions, for example, may be stronger in some 
contexts than in others, or, other teaching procedures such as higjhly 
structured textbooks or pupil-directed inquiry programs may be differ- 
entially effective in different contexts. 

At the same time that one identifies contextual variables and hopes 
that research in this area can be used to provide clearer statements on 
the generality of teaching skills or instructional programs, one notes 
that such promise is accoii5>aaied by problems. Namely, the number of 
possible contextual variables is so large that it can become extremely 
difficult to disprove the latest educational gimmick because the inno- 
vators can always claim that if an innovation has not been shown to be 
effective it is because it was tried out in an inappropriate contextual 
setting. 

The number of possible contextual variables is bewildering. Con- 
sider the following list of possible contextual variables (and the 
nus^er of possible divisions within each variable)? 
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Pupil varlAlea 

socioecimomic status 
leamlag style 

level of cognitive developaent 
sex 

ethnle background 
linguistic background 
age 

attitude toward school 

Teacher variables 

beliefs, perceptions, and personality 
stability of classroom interactions 

stabiUty in (Staining sioiUr pt^il outcomes across years 
experience 

socioeconomic status 
sex 

ethnic background 
linguistic background 
age 

Curriculum variablea 

grade level 
si&Ject area 

type of curriculum (e.g. ccmtroUed inductivn, f .«e induetivv) 
outcome mea8ure(s) 

Community variables 

location 
SBS 

ethnicity 

Organizational variablea 

school size 
classroom sice 
school organization 
classroom organization 

resources available (physical, financial, staff) 
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The nusber of coeiblnatlons of contextual variables la staggering. 
If one divided each of the sS>qvb variables into the appropriate divisions 
(e.g. two levels of sex; five types of siibject area) there wculd be a 
total of veil over two nilUon possible cells. If one attended (mly to 
a stdsset of these variables such as subject area« gr^e level, school 
locatlcm and SES there would be 270 cells. But the total of all studies 
relating dassroon behaviors to student outcomes is less than 100, and 
this number grows at the rate of only eight to twelve studies per year. 

Liscussion of contextual variables suggests that we i ', <:ntttlous 
in our generalizations and limit them to those contexts in w >f eh they 
were tested. Yet, the contexts which have been suggested anw which will 
be suggested by others encosipcss far more situations than 0^ 'sdles . 

This problem of contextual varl^les csa lead to «-Mntc4wg in oppo- 
site directions. On one hand, one hopes that most contextual variables 
can be safely disregarded and that the generalizations which are developed 
will hold across most situations. Alternatively, the generaUzatioiis 
about instructional processes and student outcomes which have been 
developed to date are not particularly strong, leading to the converse 
hope that if some contextual variables were attended to the results would 
be stronger. The effect of raising these contextual variables is to 
force a researcher or reviewer to state the limits of the generalizations 
he wici: to nr.Tce, Thus, if one wished to draw generalisations from 
the research on a teacher's use of structuring statements, for example, 
it would seem reasonable, at this time, to state the contexts to which 
such a generalization night be applied and the contexts in which such 
a generalization has not been tested. 

As examples of the use of contextual variables in research, consider 
some results on two contextual variables: social class and teacher 
stability in producing pupil achievement. In the first case the results 
st^port using the contextual variable; in the second case they- don't. 

Social class as a contextual variable . The research by Brophy and 
Evertson (1973, 1974) Illustrates how one mi^t lock at contextual 
variolas. In their studies of instructional behaviors and student 



achieveseat, Chey divided their teachers into two groups— those who 
tau^t in lower class schools and those who tau^t in sdddle class schools. 
In their analyses of both the first year and the second year data there 
were a large nin&er of ^wriables for which the aagnitude and direction of 
the correlations were different in the two ccmtexts, a sufficiently large 
nuBiber of variables to suggest that classificati<m by SES context is 
worthwhile~-8t least for reading and natheaatics gain in the second grade* 

Teacher stability as a contextual variable . Brophy and Evertson 
also believed that if they liodted their san^le to ^t of stable teachers- 
those teachers who were consistent in Staining the saae amount of student 
gain across several years->*their process-outcooe correlations would 
higher than if they had merely selected a randoa sanple of teachers. 
This hypothesis was not confirtaed-»at least for reading and nathematics 
gain in the second gri^e. 

Other contextual variables . Tuere is a need for a closer look at 
the actual la^ortance of many of the contextual variables which are 
presented as potentially ictportant. X did an infonaal look at two 
contextual variables-^grade level and ssd>ject area-~-by consulting the 
tables in ny book on teaching behavior and studcoit achievetnast (1971) . 
Each study to those tables is listed by grade level and stdjject area, 
and one can use this Inforoaticm to reorganise the tables to detemlne 
whether the overall results would be different or distinct for any grade 
level or subject area. When my students and X did this, we found » first, 
that there were a large nun^er of blanks for nost of the cooblnations 
of grade level and subject areas. Ife also found that none of the results 
which were obtained were particular to any combination of grade level or 
subject area. Thus, tiie overall results on teacher questions, for exaaqple, 
are unchanged if one divide the tables on questions according to grade 
level or subject area; the results on clarity or on task orientation 
appear significant across all grade levels and subject areas reported. 
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CurriculuB Packages aa a Contextual VarlAle 

A£ter raising the horror of a two adllion cell contextual variables 
siatrix, Z am bewildered by aqr audacity when I proceed, as X shall, to 
suggest that there is an important contextual variable which has not 
received enou^ attention: curriculuBt-materials packages, let me sketch, 
hesitantly, the arguasnt for this variable. 

First, consider the possibility that a teacher does not teach fourth 
grade reading, for exasiple. Rather, she is teaching fourth grade Maciailan 
readers, or fourth grade SRA, or Ginn, or Open Court. The instruction is 
embedded within the context of instruetimial materials. 

Consider also the possibility that a curriculum program can serve 
as a tool to enable a teacher to accomplish more than he could wit2iout 
the naterials. One example of such a eurrieulun materials package (or 
an instructional package) was developed about 1910 by Kontessori <£vans, 
1971) and is a si^ezb example of this invention. The Montessori Method 
included specific, self-correctional materials (e.g. the brown stair » 
the pink tower, the golden beads), specific instructions for teacher 
interactions with the child (e.g. vocid^ulary development had a three* 
period sequence: naming, recognition, and pronuneiatioi) , and general 
instructions for teachers (e.g. collaborative work with the child, 
avoidance of "don'ts," eootional stpport). The instructional materials* 
sequencing, provision for corrective feedback, and specific and general 
instructions to teachers contained in the Montessori materials are quite 
dif fesent from the usual practice of providing a teacher with only a set 
of books, a syllabus, and vague ^Jectives. The major advantage of the 
Mmttessori package, or any curriculum materials package, is that it may 
enable a teacher to accomplish ends which would not be accomplished 
without these materials. 

The advantage claimed fcr this approach is that curriculum-materials 
packages represent potential experimental treatments and also provide a 
tftacher with means co accomplish more than he could without the materials, 
imather these materials and instruetiaas are suiteble, whether they are 
uscil properly, and whether the outcomes are the cmes which are expected, 
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asa the risaarch quastLnis. Althou^ thera haa baen ouch dayalopamt of 
curriculum tnaterlals packages, there has been relatively little study ot 
the research questions. 

The Sullivan prograoned reading nate* whitih are used in kinder- 
garten and first grade as part of two FU^^d Variation Follow Through 
(PVFT) programs-^-The Pittsburgh model and the Kansas aodel— are additional 
examples of facilitative materials which provide halp for a teacher. 
Among the activities handled in this program are: locating materials, 
sequencing instruction, and providing feedback to children. The raspon* 
sibility for developing the instruction and testing the efficacy of the 
approach lies with the developer who has, presumably, solved many of 
these instructional problems in previous trials. 

The results of the coopariscaa of effects of PVFT programs are st-<.ll 
being analyssed and reanalyzed, but one conclusion which usually appears 
in these reports (e.g. Stallings, 1973; Stallings fsad Rascovits, 1974) is 
that the structured programs, such as the Pittsburgjb and the Kansas model, 
. yield greater pupil achievement in reading and math than do the more 
flexible programs. The likely cause for this superiority would appear to 
be the materials and the instructions for their use. It seems unlikely— 
at least at this time-~>that the different PVFT programs differ in other 
variables such as teacher expectations, teacher selection, teacher abilities, 
or teacher training, and the conclusion that Z currently prefer is that 
the differences in program effects in reading and math are due to the types 
of materials used and the instructions for their use. Such a conclusion 
i^out the facilitative effect of some curriculum programs needs to be 
tested and modi.fied, and one purpose of this paper is to encourage such 
testing and modification. 

One ii^lication of this idea is that more time might be devoted to 
evaluating and testing the instructional materials which are chosen and 
used in classrooms. Such materials include not only well organiaed 
curriculum packages, but textbooks as well. 
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Xdentifyiiig Curriculum EXenents 

If sone curriculums are more effective than others for similar 
aads, then learning vhy they are more effective mi^t facilitate both 
curriculum development find curriculum modification. Presumably differ- 
ences in content and ^hasis will be part of the answer, but what 
these other elements are remains to be determined. Some possibto vari* 
ables are: 

The organization of the materials for study 
The questions which are provided for the teacher (e.g. pro- 
vided in worldtiooks on study sheets for the teacher) 

The feedback which materials provide for students. 
Although apparently there has not been much work on the identifi- 
cation of the functions which materials play in facilitating instruction, 
the broad outlines of possible functional variables are present in any 
tesctbook on Instruction. These functions Include structuring learning, 
presenting material to be learned, soliciting respcmses, reinfordng 
and correcting, extending, promoting practice, maintaining assessing 
pupil learning, organizing and sequencing instruction, and pro\4di.ng 
for consolidation of learning. 

If curricul m) materials were analyzed in terms of the above list 
or a similar list, one might be able to suggest that some packages provide 
more tools for learning than do other packages. As an example, consider 
how materials can provide feedback to pt:^ils. If piqtlls \Aio are attempting 
to improve their reading comprehension use materials which are accoi»- 
panied by multiple choice questions, they may obtain more feedback on 
their ability to understand a story than they would obtain through a 
class discussion. Furthermore ^ if the multiple choice questions were 
(consistently organized by type so that a pt^il learned whether he was 
having particular trouble figuring out words in context or getting the 
viain idea of a paragraph, then such organisation may provide more feed- 
back than multiple choice questions alone. 

Xn contrast to the t»ual emphasis tq^on coding interacticm, the 
esiphasls in the above excmples is vpoa materials ax.d determining whether 



or how such maurials help both the teacher and the pupil. Current 
obeervational procedures may obscure the usefulness of naterials. For 
exanple, coding the types of questions which teachers ask may obscure the 
possibility that the teachers had to develop their own questicms in <me 
curriculum and were provided with both specific and illustrative questions 
in another curriculum. Similarly, the organization and maintenance of 
student practice may be well developed through structured materials in 
oca curriculum and left up to the teacher's devising in another. 
Rrnediation exercises for pupils may be built into short, instructional 
p&ckages in one curriculum, whereas in another, the teacher may be 
responsible for developing and organizing remediation naterials. 

The above examples ate merely suggestive of research. Of course, 
one would not expect that all multiple choice questions in aU curriculum 
packa^ are of equal merit, end the effects of different types of written 
questions has long been a topic of study. The point, however, is that 
the usual studies of classroom interaction have t^ded to focus cmly upon 
the questions which teachers or pupils asked and have not included anal- 
ysis of the kind, sequence, and format of questions which the materials 
asked pupils. Whether such additional research into instructional elements 
present in the materials will be functional is unknown at present. My 
argument, however, is that the instructional variables inherent in 
materials should be included in the study of classroom learning. 

Using Curriculum in Process-Outcome Research 

This section contains some sug^stions and isst^s in including 
knowledge of curriculum in process-outcome research. The ideas were 
developed from an inspection of current practices and are expanded and 
modified from an earlier piece on this topic (Rosenshine and Furst, 1973). 

The first question might be the extent to which knowledge of curric- 
ulum predicted piqpil achievenent. That is, are there some packages or 
cocibinatiotts of packages which are more effective in reaching the 
criteria than others. If the question is answered positively, how does 



one then look at teacher behavior? Does one look at teacher behavior 
across curricula or does one only look within curricula? 

There are at least three types of questions one can ask about dif- 
ferent currlculun prograoss; 

1* Are soaie curriculum packages (or combinations of packages) 
more effective than others on reaching the same goals? 

2. Zf some programs are differentially effective, then are there 
general rules about effective . .sn«nts across different packages? 
Are there soma teacher behaviors which correlate with pupil 
outcomes across packages? Are there elements in the curriculum 
materials which are associated with greater pupil achievement? 
Some elements which mi^t be considered are among those suggested 
in the previous section: 

The questions which are offered for the teacher to ask 

The organization of the materials for study 

The feedback which materials provide to students. 

3. TJhat are correlates and what are causes of outcomes within 
individual programs? What modifications in materials, organi- 
zation and use of materials, and teacher behavior might facilitate 
the effectiveness of individual programs? 

1. Comparison of outcomes 

There are a nus^er of possible steps in studying whether these 
different programs and cosdbinations of programs are yielding different 
effects. One procedure*-which was used by Soar (1971) to stu<fy processes 
but which seems applicable to outcomes also— would be to group the 
programs by type and use something like a multiple-range test to deter-* 
mine which groupings of programs are similar as to outcomes. When 
programs do not differ as to outcomes, then it would seem fruitful to 
use knowledge of curriculum in the subsequent analyses. 

If programs were found to differ in effects, two additional analyses 
on effects seem relevant to aid in interpreting these results. The first 
analysis would look at effect by level of Implementation. 
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Bffacts and level of Implenactatlcn . A najor question vithia th« 
stu^y of any curricuXuB package la tiie relationship between eiirrlcttlt» 
effects and level of implementation. Level of inpleaentation zefers to 
whether a program or 4£ package is being implemented according to the 
specifications of the developers. The research questim is whether levels 
of io^lementation (or levels of different inpleaentation variables) affect 
the outcomes. That is» for some programs the desired effects nay not 
occur unless the program is weU-ioplemented, whereas other prograas nay 
be so robust that when poorly implementei they may be as effective as 
when well-implemented. On one hand, then, it can be argued that it doesn't 
make sense to compare the effects of well-implemented versions of ptograa A 
against poorly implemented versions of program B. On the other hand, it 
is also possible that for program B the implementation variables are not 
so important as we thou^t asd that a version labeled as poorly inpleaented 
is as effective as a version labeled as well-implentt»ted. 

In any discussion of the effects of curriculum packages, one has to 
note that in the First Year Reading Study (Bond and Dykstra, 1967) in 
which a variety of reading approaches and materials for teaching these 
^preaches were cellared, the general result was that me approach or 
set of materials was not consistently superior to another. A ninter of 
possible explanations for these results should be considered. One explana^ 
tion could be that the curriculum programs did not differ between thewielves 
in content covered or c<mtent emphasized, and this lack of difference was 
reflected in the results. Another possibility is that the results cannot 
be adequately interpreted without information on program inpleaentation. 
Sone programs may have been more easy to implem^t than others, and degree 
of implementation may have had different importance in different programs. 
Xt is thus suggested that without data on implementation (and on the impor- 
tance of is^lementation) comparisons between programs n^ be preaature. 

XmpleoentatiOT may be a critical variable in some prograis and not 
in others. In one renovt (Stallings and Kascovitz, 1974), correlations 
were cooputed between the total implementation score in each of the class- 
rooms within each of the Planned Variation Follow Ihrougli (PVFT) and the 



-X2- 

adjusted measures o£ pupil gain in reading and oathematics. The results 
did not follow any pattern. In most cases the eorrelations were loir 
(.0 to .2), and In those eases in which results were .3 or better, the 
overall pattern was not consistent across grades or st&ject area within 
the FT model being studied. 

Information on level of isq^lementatlon can be used a number of ways. 
One procedure would bci to limit comparisons of outcomes to those classrooms 
which are labeled as high io^lementers. A subsequent analysis can deter- 
mine leather level of implementation is an important variable in different 
programs. In cases in which level of implementation Is not as Important 
as was thought, then all classrooms using a program can be Included In 
the analysis. 

(Research using level of implementation variables will continue to 
be difficult because in addition to studying the effects of overall 
measures of in^tlementation one would also need to study the effects of 
Individual implementation variables.) 

2 . petermlnlng correlations across programs. 

In studies within Planned Variation Follow Through (Soar, 1971; 
S tailings and Kaskovltz, 1974) correlations between observed behaviors 
and pt^ll outcomes were computed across the seven or eight programs 
without regard for the curriculum in which the behaviors to<^ place. 
Thus, variables such as the kind of questions teachers asked, the amount 
of time pt^lls spent on academic work, the amount of initiation pt^lls 
did, and the types of responses teachers make to ptq^lls were coded and 
the frequencies of these variables were correlated with the adjusted 
outc(»ie measures without any consideration of the program in which these 
took place. 

The authors of these reports have interpreted such results as giving 
general information on the relationship of the process variables and 
student outcomes. Although I would like to accept their results as such, 
there are several questions and problems which are still unanswered. 
First, if the programs are being well implemented, then teacher behaviors 
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wlthin programs should vary very little. That is, one wo\ild expect that 
the teacher behavior of elaborattog on a ^vptl initiat'^a idea would be 
infrequent in the structured prograias and fairly frequent in the flexible 
prograsis with little variation within classrooms within a structured or 
a flexible program. If there is a strong correlation between teacher 
elaboration and pxspil outcomes, such a cort elation may represent differ- 
ences in programs and not independent behavioral variables whidi are 
related to pupil outcomes. Thus, once again there is an implication of 
instances in which it is the curriculum and not the instruction per se 
that makes a difference. 

3. Determining within program correlations. 

Research within curriculum singly involves searching for correlates 
of ptqpil differential outcomes associated with a particular curriculum 
or instructional package. Such a search could take place by manipulating 
potentially important instructicmal vari£ibles and noting the ef fectr> of 
the manipulation, or it could take place by observing natural variation 
in instruction which occurs as a curriculum is being implemented and 
ascertaining which, if any, elements of this natural variation are 
related to the outcomes. Hopefully the two processes feed into each 
other, with the results of the esiperimental studies suggesting new 
correlational variables, and the results of the correlational studies 
suggesting new experim<3ntal variables. 

In the unresolved area of how to analyze and interpret results 
across programs, it would seem appropriate to conduct more research on 
correlates and causes of effectiveness within programs. Siq;>pose that 
some teachers were using ^!c6raw Hill materials for teaching reading 
comprehension, while other teachers were using the Bamell Loft Specific 
Skill Series and still others were using the Science Research Associates 
(SRA) reading boxes. Even if one identified correlates of outcomes across 
programs, one would not know if such variables iterated as correlates 
within these different programs. Becat^e one wishes to apply results to 
specific programs, it would seem relevant to study bow these variables 
operate within specific programs. 
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If the curricula yield different effects, then one can wonder 
whether It worth c«atlnuing to look at teacher behaviors across currlcu- 
luas. That Is, If dassrooos using the Specific Skills Series attained 
higher reading achievement than was obtained in other packages, looking 
at differential teacher interactive behavior may be inappropriate because 
in this series most of the work is done individually and children are 
able to check their answers quickly and move to remedial work or higher 
level work. In such a structured situaticm, there is minimal need for 
teacher-pupil interactions, and therefore it is likely that there would 
be fwer displays of teacher presentations, teacher enthusiasm, or higher 
order questioning. The more appropriate teacher behaviors would appear 
to be more macageilal than presentational. 

At the same time, how does one look at a teacher's assignment of 
remedial work if we kacsg that some classrooms are using the Specific 
SklU Series? It is possible that the behaviors of assigning remedial 
work will not differ greatly in frequency across curriculum programs, 
but such assignment may mean something different when it takes place 
within the context of the Specific SklU Series. 

Ihus, regardless of how one feels about the importance of accepting 
curriculum as a contextual variable in the gf leral study of die effects 
of teaching, it would seem Important to study teacher behaviors and their 
effects within curriculum packages. Although some research has been 
conducted within programs (e.g. Kochendorfer, 1967; Siegal and Rosenshlne, 
1972; Siegal, 1973) there has been relatively little research of this 
type, despite its apparent utility for improving specific programs. 
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The Relavance of Content Covered and Content ^nphasized 

The relevance of content covered in assessing curriculum effects 
is an area which needs amch taote research. On <me side, it can be argued 
that what is tau^t is nore important than how it is taught, and evidence 
on this point will be presented in this section. But if this argument is 
pushed, one could claim that the results of any curriculum package cannot 
be compared with another package be.'.ause each package is en^hasizing 
different criteria. 

Program content and program emphasis. Based on their ciniq>ari8on of 
results of twenty'^ three studies ^ich c«spared innovative and traditional 
curriculums, Valker and Schaffarick (1974) concluded that different 
curriculums produced different patterns of achievement, and suggested 
that these patterns are strongly influenced by the content that is included 
and enq^hasized within each. They assert that because different curriculums 
in the same subject area emphasize different content and skills to be 
learned, one does not gain much from comparing these curriculums. 
Furthermore, they argue that against such powerful variables as content 
and emphasis, other variations such as teaching procedures have relatively 
less influence on outcomes. They are, of course, aware of the limits of 
their conclusion. "Hie would be foolish to expect, for example, that 
simply including more content in texts and tests would ensure ever greater 
knowledge and skill." And, they are aware that their conclusion exists 
in the aggregate and is limited to patterns of achievement in large groups 
of students studying roughly similar kinds and amounts of content. 

To be sure, they present their claim as an idea to be tested rather 
than as an accompUshed fact. They suggest, furthermore, that different 
sorts of presentations of an identical item of content will no dotibt 
produce differences in achievement and recommend further research to 
determine why certain ingredients contribute to a successful curriculum 
presentation. 

One approach to the question of effective teaching mig^t be to 
determine first the influence of three variables: content covered and 
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es^hasized, time available ior inatrttetion, ibid student attention to 
task, l^e hypothesis would be that these variables are ii»re powerful 
than any instructional variables. If the above hypothesis can be demon- 
strated, the next step would be *hat other variables, if any, have 
impact. 

International variations in content covered. It is also possible 
to suggest the effects of variations in content in different countries. 
Perh^s the most dramatic sttuiy of the io^ortance of content was conducted 
by Pid^on (1970). In this study, the mathematics achievement of 3,000 
eleven-year-old piipils in CaUfomia was coii9>ared with that of a similarly 
stratified sas^le of 3,000 eleven-year-old pupils in England. XWo 
thousand English pupils achieved a score above 35 <m the 75 item test, 
whereas mly 54 (sic) Calif omld pupils achieved a score above 35. 
Fidgeon (1970) inspected the mathematics textbooks used in the two 
countries and concluded that the material in the material in the English 
textbooks was one to two years in advance of the material in the Cali- 
fornia textbooks. He concluded that the English pupils were able to 
learn more mathematics because the teachers expected more learning from 
them, and these teacher expectations were reflected in the content that 
was covered. 

Additional evidence on the io^ortance of content comes from two 
studies conducted by the International Association for the Evaluation 
of Educational Achievement (lEA). In two cross-national studies, one 
in mathematics and one in science, the teachers were asked to look at 
the items on the test and indicate the percentage of students in the 
class who had the opportunity to learn the material covered by the 
item. When these scores were asses^led by country there was a strong 
positive correlation between opportunity to learn material and actual 
achievement scores by country. (When correlations were computed be- 
tifeen opportunity to learn in individual schools and student achieve- 

within country the correlations were not significant, possibly 
because of limited variation within each country) . One could argue 
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that Che contfmt which is covered Indicates the nati<mal expectancy. 

Within course research. It is also possible to note the is^or- 
tanee of content and en^hasis iq)on learning the ecmtent by looking 
at studies which took place within a cttrriculum or within a course 
of study. Ros^ishine (1968); Shutes (1970); and Chang and Raths (1971) 
all found significant relatioiships between aa assesso^it of the content 
or skills covered and pupil achievement. Stallings end Kaskovitz 
(1974) found that class activities involving nusnbers, nus&er of children 
involved in saathematics , and total vQiAaX interactions regarding oath 
were all significsntly and positively correlated with math gain in both 
the first and third grade; number of activities involving reading and 
nusber of children involved in reading was significantly and positively 
correlated with reading gain in the first grade but not in the third 
grade. 

As part- of their two*year stu4y of process-outcome relationships 
in 31 first and 28 secoad grade classrooms, Brophy and Evertson (1974) 
asked observers to estimate the percentage of time that teachers devoted 
to various curriculum areas. Although some positive correlations were 
obtained, the correlations between time in math, or time in reading, 
and appropriate measures of ptq}il gain were not consistent or strong 
across each of the two years of their stu^. However, in their sunmary 
of all their results, Brophy and Evertson concluded that the teachers 
who had the most success were those who were most determined to ti^ach 
the students, who took responsibility for making sure that pupils learn, 
and who were determined to overcome instructional obstacles. The teachers 
who got the greatest gains did so by "overteachlng," by presenting mate- 
rial in small doses, with greater repetition and opportunity for practice 
than less successful teachers. 

Another approach to the importance of cmitent and teacher ability 
to cover appropriate content might be derived from the study by 
Pellegrini and Hicks (1972). In this study, tutors worked individually 
with elementary school children. Three groups provided cot^arisons to 
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assess the ei^etancy txeatnent: One group of tutora was told their 
tuteea vexe of hi^ ability and would make dramatic gains in academie 
areas in the neict few iconths; a second group was told their students 
ware of average ability; and a third grovqp was told their children 
were below average in ability. After 17 weeks of tutoring* the diildren 
were retested on the PPVT and the WICS Similiarities Test. There were 
no siffiificant differences between these three tseatoent groups. A 
fourth group was also told that their students were of high ability, 
and this set of tutora waa shown saaqple itens from P?VT and WICS Sini-* 
larities Test. This group scored signfiicantly hi^er than the other 
three on the FPVT probably because the tutors' familiarity with the 
ittstrunent resulted in instruction in the discrete skills necessary 
for successful test performance. While this group did not perform 
better than the others on the ITXSC* the tutors probably did not possess 
sufficient skills to teach the more complex reasoning required by itens 
on this test. 

A series of studies involving trainees tutoring pupils for periods 
of thirty minutes or less have also yielded mixed results. In these 
studies (Beez, 1968; Carter, 1969; Panda and Guskin, 1970; and Brown, 
1970) the tutors were given fictitious information about their tutees 
which was desired to induce a set for a low achieving or a hi^ 
achieving pupil. In three of the studies (Beea, 1968; Carter, 1969; 
snd Brown, 1970) the teachers attempted to teach more words to the 
"high achieving pupils." In the fourth study (Panda and Guskin, 1970) 
there was no difference in the ttuad>er of words taught. The results 
on the nuniber of words learned we^ fairly consistent; students learned 
more words in two (Beez, 1968; Carter, 1969) of the three studies in 

i^ch they were taughc more words. 

In summary, the results of some of the above, studies suggest 
that content covered, opportunity to learn, and a teacher's emphasis 
upon student achievement are important variables for continued stud^. 
One wishes that there were more studies on this topic and that the 
results of the above studies were stronger. 



What is tau^t and how It is taught* If content covered is as 
iaportaat as the results appear to indicate, it seems worthwhile to 
prdbe the limits o£ a proposition such as 'Hdiat is taught is more ta^ 
portant than how it is taught." Walker and Schafferick suggest that 
one cannot expect that singly including more content in texts and tests 
would ensure greater knowledge and sldll. Furthermore, we have learned 
from htsodreds of experimental studies that different forms of pTOsen- 
tation of identical content have frequently led to differences in achieve-' 
sent. 

Althou^ the results on content covered are strong, these correl-* 
ati<m8 are seldom above .5 and the resultt? are not always significant 
or consistent. Overall content covered appears to be an extremely 
important variable which has not received sufficient attention in 
research on teaching effects. Furthermore, it remains to be determined 
which procedures for teaching identical content and skills are most 
effective. However, it is suggested that previous research has been 
overly concerned with how something is taught and has uot spent enough 
time identifying what has been taught. Perhaps future work based on 
observation of different classrocms could first establish what has 
been taught, then loc^ at how it hss been tau^t, and then look at 
the interactions between them. 

One place where one could probe whether knowledge of how something 
Is tau^t adds to what is taught might be throu^ inspection of the 
results of the Planned Variation Follow Through research. The programs 
in PVFT range from strongly structured programs such as DZSXAR to 
different forms of flexible and open classrooss. The results of these 
programs are not final and one expects that conclusions will be revised 
as new data come in and old data are interpreted, but the results to 
date are that the more structured programs are consistently obtaining 
greater pupil achievement in reading and mathematics than the more 
flexible prograxffis. Assuming for the moment that these n»8ults will 
hold up (and although there has been consistent evidence favoring 
structured, high content esphasis programs for low income children tma 
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shouXd want to inspect the ebaracteristlee o£ isolated flexible dass- 
rooos which are e<i'aally strong in producing reading and madi aehievesient) » 
there are at least two possible reasons for the greater effectiveness 
of the structured programs: 

1. The structured programs are emphasizing different ends (or 
different patterns of ends) from the flexible programs, or 
2* Givea similar ends, the materials and instructions for using 
the materials in the structured programs are superior to the 
materials and instructions in the flexible programs. 
Bach of these two possibilities might be probed in future research* 

Prctelng the importance of content covered. One conclusion from 
the research cited above is that when instructional programs differ in 
the content which is presented and emphasized, these differences are 
reflected in student achievement scores. Furthesmore, within an in- 
structlimal program, variations in content covered are frequently 
positively and significantly correlated with student achievement, 
although these correlations are neither as large or as consistent 
as comparisons made across different programs. 

Walker and Scbaffarick (1974) have noted that altbou^ the Above 
conclusions appear obviocs, such <^viousness is not reflected in deci- 
sions about curriculum packages and textbooks. Furthermore, the obvi- 
ousness of considering content has not been reflected in the analysis 
of process-outcome studies or in the items on teacher-evaluation check- 
lists. 

Other content variables. There are two other variables which 
would seem related to concent covered, variables which are covered 
in Carroll's (1963) model of school learning but which receive insuf- 
ficient attention In research programs. One such variable is the amount 
of attention a ptspll gives to the task. It would seem worthwhile to 
conduct more research on the contributions of these variables to student 
learning. 
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Another approach to the Inportance of content covered la suggested 
by Duhin and Taveggia CX968) in their review of studies coaq^aring the 
effects of different nethods of college teaching. Hhe teaching methods 
conpared included lecturing, discussion » supervised independent study, 
lecture-discussion, and unsupervised independent study. DvS>in and 
Taveggia concluded that these dlffentnt instructional methods did not 
produce differences in stiklent achievement. 

In discussing the results of their review. Dub in and Taveggia 
suggest thAt research a&»ng cos^arative teaching methods should first 
focus on the question "Uhat is there that is the same about two teaching 
methods?" They suggest that the textbook (s) utilized is an outstanding 
commonalty in classes tau^t by different methods and that various 
methods of instruction have only minor intact conpared to the inpact 
of instruction. They further recommend that future studies should 
examine differences among textbooks rather than among teaching methods. 

I interpret their conclusion as saying that when students have 
access to the same content and know what they are expected to learn 
(and when they are capable of learning from the textbook) then differ- 
ences in instruction will have relatively little impact. 

Eliminating Alternative Hypoti) ses 

A nus&er of those who have been reviewing research on teadiing 
have suggested that, although the results to date have not been as 
strcaog as we hoped, results will be better if future research incor- 
porates their latest ideas (Brophy and Good, 1974; Good and Brophy, 
1973; Duncan and Biddle, 1974; Rosenshlne and Furst, 1971, 1973). 

The number of suggestions is extremely large. One wishes that 
we would spend more time eliminating unproductive ideas. X would 
hope that future studies could begin by explicitly slating why the 
author hoped that the results of his/her study would be stronger than 
the results of previous studies, and that the conclusion of the stud^ 
would evaluate the merit of the author's new ideas. Unfortunately, 



if A new idea U- not validated by the study, the* tendency is to fault 
the circunstances of the atudy and retain the original idea without 
laodlflcatlott. 

One of the axeaa for which there are a nua^er of aufigeatlone la 
In how to code elaasroon transactions. Cooaaenta have been made as to 
the coding unit (e.g. tine unit, natural break in purpose, sign unit), 
the nuBber of eo-varlablea to be Included with an Interacticm (e.g, 
content, speaker, level of conceptualization, nunber of students 
attending) , whether previous events should be Included In the coding 
unit, and the size of the variable In the unit (e.g. should Instances 
of praise be subdivided as to type). One wishes that when soneone su^ 
gests these new approachea (and, belatedly, I Include iqrself) he would 
also discuss the types of results which would Indicate that the new 
approach does not have merit. 
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